Weaknesses: Theoretical: The authors provide little formal justification for their approach. One of the main contributions seems to be the increase of effective sample size by performing data augmentation. What is unclear is why the increase actually happens. In remark 3.1, the authors attempt to answer the question "How much data can we generate using model-free CoDA?", claiming an exponential increase in data. This fact is not immediately clear.

Reviewers were positive and excited about the paper, and I agree with the general sentiment that the work is a significant step in the right direction. Having said that, there are some issues that I would like to see fixed to make its final version more comfortable to read, sound, consistent, and well-positioned regarding the broader literature. Towards this goal, first, read the reviews carefully and try to incorporate their feedback as much as you can. I will list some critical issues below, mostly in addition to the ones raised by the reviewers. Please, re-define causal model to account for the bipartite structure mentioned in the rebuttal; that's a strong constraint over the SCM-space but appears to be enough for the paper's purposes.

Many dynamic processes, including common scenarios in robotic control and reinforcement learning (RL), involve a set of interacting subprocesses. Though the subprocesses are not independent, their interactions are often sparse, and the dynamics at any given time step can often be decomposed into locally independent} causal mechanisms. Such local causal structures can be leveraged to improve the sample efficiency of sequence prediction and off-policy reinforcement learning. We formalize this by introducing local causal models (LCMs), which are induced from a global causal model by conditioning on a subset of the state space. We propose an approach to inferring these structures given an object-oriented state representation, as well as a novel algorithm for Counterfactual Data Augmentation (CoDA).